Estimating recruitment rates for routine use of patient reported outcome measures and the impact on provider comparisons

BACKGROUND The routine use of patient reported outcome measures (PROMs) aims to compare providers as regards the clinical need of their patients and their outcome. Simple methods of estimating recruitment rates based on aggregated data may be inaccurate. Our objectives were to: use patient-level linked data to evaluate these estimates; produce revised estimates of national and providers' recruitment rates; and explore whether or not recruitment bias exists. METHODS Case study based on patients who were eligible to participate in the English National PROMs Programme for elective surgery (hip and knee replacement, groin hernia repair, varicose vein surgery) using data from pre-operative questionnaires and Hospital Episode Statistics. Data were linked to determine: the eligibility for including operations; eligibility of date of surgery; duplicate questionnaires; cancelled operations; correct assignment to provider. Influence of patient characteristics on recruitment rates were investigated. RESULTS National recruitment rates based on aggregated data over-estimated the true rate because of the inclusion of ineligible operations (from 1.9% - 7.0% depending on operation) and operations being cancelled (1.9% - 3.6%). Estimates of national recruitment rates using inclusion criteria based on patient-level linked data were lower than those based on simple methods (eg hip replacement was 73% rather than 78%).Estimates of provider's recruitment rates based on aggregated data were also adversely affected by attributing patients to the wrong provider (2.4% - 4.9%). Use of linked data eliminated all estimates of over 100% recruitment, though providers still showed a wide range of rates.While the principal determinant of recruitment rates was the provider, some patients' socio-demographic characteristics had an influence on non-recruitment: non-white (Adjusted Odds Ratio 1.25-1.67, depending on operation); most deprived socio-economic group (OR 1.11-1.23); aged over 75 years (OR 1.28-1.79). However, there was no statistically significant association between providers' recruitment rates and patients' pre-operative clinical need. CONCLUSIONS Accurate recruitment rates require the use of linked data to establish consistent inclusion criteria for numerators and denominators. Non-recruitment will bias comparisons of providers' pre-operative case-mix and may bias comparisons of outcomes if unmeasured confounders are not evenly distributed between providers. It is important, therefore, to strive for high recruitment rates.